{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MNIST Image Classification with TensorFlow\n",
    "\n",
    "This notebook demonstrates how to implement a simple linear image model on [MNIST](http://yann.lecun.com/exdb/mnist/) using the [tf.keras API](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras). It builds the foundation for this <a href=\"https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/image_classification/labs/2_mnist_models.ipynb\">companion notebook</a>, which explores tackling the same problem with other types of models such as DNN and CNN.\n",
    "\n",
    "## Learning objectives\n",
    "1. Know how to read and display image data.\n",
    "2. Know how to find incorrect predictions to analyze the model.\n",
    "3. Visually see how computers see images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "!sudo chown -R jupyter:jupyter /home/jupyter/training-data-analyst"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.6.5\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import shutil\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow.keras import Sequential\n",
    "from tensorflow.keras.callbacks import ModelCheckpoint, TensorBoard\n",
    "from tensorflow.keras.layers import Dense, Flatten, Softmax\n",
    "\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploring the data\n",
    "\n",
    "The MNIST dataset is already included in tensorflow through the keras datasets module. Let's load it and get a sense of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
      "11493376/11490434 [==============================] - 0s 0us/step\n",
      "11501568/11490434 [==============================] - 0s 0us/step\n"
     ]
    }
   ],
   "source": [
    "mnist = tf.keras.datasets.mnist.load_data()\n",
    "(x_train, y_train), (x_test, y_test) = mnist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Image height x width is 28 x 28\n",
      "There are 10 classes\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-01-03 16:46:45.352741: I tensorflow/core/common_runtime/process_util.cc:146] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.\n"
     ]
    }
   ],
   "source": [
    "HEIGHT, WIDTH = x_train[0].shape\n",
    "NCLASSES = tf.size(tf.unique(y_train).y)\n",
    "print(\"Image height x width is\", HEIGHT, \"x\", WIDTH)\n",
    "tf.print(\"There are\", NCLASSES, \"classes\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each image is 28 x 28 pixels and represents a digit from 0 to 9. These images are black and white, so each pixel is a value from 0 (white) to 255 (black). Raw numbers can be hard to interpret sometimes, so we can plot the values to see the handwritten digit as an image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The label for image number 12 is 9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAbrklEQVR4nO3df3BUdbrn8U8HSAOadAwx6WQIGFBhFAhXBjIpFFFSQKzigrBbom4tuCyUTrAGMv6ozCioM1WZwSqH0s3g1u4I4y0Rh71CrtQtajWQcB0TXCIsl1FTJJsRWJIwsks6BAmBfPcP1nYaAnia7jzp8H5VnSrSfb45D8cu3zmkc+JzzjkBANDHkqwHAADcmAgQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwMdh6gEv19PTo+PHjSklJkc/nsx4HAOCRc04dHR3KyclRUtKVr3P6XYCOHz+u3Nxc6zEAANfp6NGjGjly5BWf73cBSklJkSTdq4c0WEOMpwEAeHVe3fpY/xz+//mVxC1AFRUVevXVV9Xa2qr8/Hy98cYbmjZt2jXXffvPboM1RIN9BAgAEs7/v8Potb6NEpc3Ibz33nsqLS3V2rVr9dlnnyk/P19z5szRiRMn4nE4AEACikuAXnvtNS1fvlxPPPGE7rrrLr355psaPny43nrrrXgcDgCQgGIeoHPnzqm+vl5FRUXfHSQpSUVFRaqtrb1s/66uLoVCoYgNADDwxTxAX3/9tS5cuKCsrKyIx7OystTa2nrZ/uXl5QoEAuGNd8ABwI3B/AdRy8rK1N7eHt6OHj1qPRIAoA/E/F1wGRkZGjRokNra2iIeb2trUzAYvGx/v98vv98f6zEAAP1czK+AkpOTNWXKFFVVVYUf6+npUVVVlQoLC2N9OABAgorLzwGVlpZqyZIl+tGPfqRp06Zp/fr16uzs1BNPPBGPwwEAElBcAvTII4/or3/9q9asWaPW1lZNnjxZO3fuvOyNCQCAG5fPOeesh/hboVBIgUBAMzWfOyEAQAI677pVrUq1t7crNTX1ivuZvwsOAHBjIkAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJmIeoJdeekk+ny9iGz9+fKwPAwBIcIPj8UnvvvtuffTRR98dZHBcDgMASGBxKcPgwYMVDAbj8akBAANEXL4HdPjwYeXk5GjMmDF6/PHHdeTIkSvu29XVpVAoFLEBAAa+mAeooKBAmzZt0s6dO7VhwwY1NzfrvvvuU0dHR6/7l5eXKxAIhLfc3NxYjwQA6Id8zjkXzwOcOnVKo0eP1muvvaZly5Zd9nxXV5e6urrCH4dCIeXm5mqm5muwb0g8RwMAxMF5161qVaq9vV2pqalX3C/u7w5IS0vTnXfeqcbGxl6f9/v98vv98R4DANDPxP3ngE6fPq2mpiZlZ2fH+1AAgAQS8wA988wzqqmp0V/+8hd98sknevjhhzVo0CA9+uijsT4UACCBxfyf4I4dO6ZHH31UJ0+e1K233qp7771XdXV1uvXWW2N9KABAAot5gLZs2RLrTwkAGIC4FxwAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYCLuv5AOSCQXZt7jec3gNW2e13ww7p88rxniG+R5Tbe74HmNJE0/sNjzmhG/8P4bjH1/+d+e15ycd5fnNenbD3leI0k9HR1RrcP3wxUQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATHA3bPR7Pr/f85qOv58c1bHWlr/lec39w854XtPjeYXU7byv6YnqSNK/TN7sec09Ly71vCY/6P1r4Mrb/pPnNVPTnva8RpKy3vgkqnX4frgCAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMcDNS9HtdMyd6XrNrvfcbVkZr9zc3e16z5lf/wfOaIWeiuBtplEKjvX9tmuz9nqx67hnvN39t7znvec3NLRc8r0H8cQUEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJjgZqToU64w3/Oa8g3/OQ6T9O7Rpoc8rwmtzfW85pbdtZ7X9KXA7Xme10ze2uR5zQ+TvX8NPL5ytec1d/63vZ7XIP64AgIAmCBAAAATngO0Z88ezZs3Tzk5OfL5fNq+fXvE8845rVmzRtnZ2Ro2bJiKiop0+PDhWM0LABggPAeos7NT+fn5qqio6PX5devW6fXXX9ebb76pvXv36qabbtKcOXN09uzZ6x4WADBweH4TQnFxsYqLi3t9zjmn9evX64UXXtD8+fMlSW+//baysrK0fft2LV68+PqmBQAMGDH9HlBzc7NaW1tVVFQUfiwQCKigoEC1tb2/66erq0uhUChiAwAMfDENUGtrqyQpKysr4vGsrKzwc5cqLy9XIBAIb7m53t/SCgBIPObvgisrK1N7e3t4O3r0qPVIAIA+ENMABYNBSVJbW1vE421tbeHnLuX3+5WamhqxAQAGvpgGKC8vT8FgUFVVVeHHQqGQ9u7dq8LCwlgeCgCQ4Dy/C+706dNqbGwMf9zc3KwDBw4oPT1do0aN0qpVq/SrX/1Kd9xxh/Ly8vTiiy8qJydHCxYsiOXcAIAE5zlA+/bt0wMPPBD+uLS0VJK0ZMkSbdq0Sc8995w6Ozu1YsUKnTp1Svfee6927typoUOHxm5qAEDC8xygmTNnyjl3xed9Pp9eeeUVvfLKK9c1GAam//uLbzyvmeL3fpyHvlzofZGkQc94/x7koP2fRXWs/uzUlKxr73SJtZl/jMMkl8v9731yGPQB83fBAQBuTAQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDh+W7YwLeat0zyvObPf7fR85pj573fQTvpF7d4XiNJbv/BqNb1Vz5/FLcSl3T7qs89r0mK4uvZJ76a5XnNsO2fel6D/okrIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABDcjRdT+/V3ebwrZox7Pa746n+p5jeoG1k1FpehuLNqwPj+qY1WOqvC8xvt/WemrV8d5XjNce6M4EvojroAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABPcjBQwMOhu7zfh/OLpgOc1X87zflPRaO3+5mbPa1I+afa85oLnFeivuAICAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAExwM1JE7R+bJ3te8+yIf/W85u/8nZ7X3HfwrOc1fWna8Pc9r3lgmPe/U4/nFdH72f/8N57XjGz7cxwmQaLgCggAYIIAAQBMeA7Qnj17NG/ePOXk5Mjn82n79u0Rzy9dulQ+ny9imzt3bqzmBQAMEJ4D1NnZqfz8fFVUXPkXXc2dO1ctLS3h7d13372uIQEAA4/nNyEUFxeruLj4qvv4/X4Fg8GohwIADHxx+R5QdXW1MjMzNW7cOD311FM6efLkFfft6upSKBSK2AAAA1/MAzR37ly9/fbbqqqq0m9+8xvV1NSouLhYFy70/pvcy8vLFQgEwltubm6sRwIA9EMx/zmgxYsXh/88ceJETZo0SWPHjlV1dbVmzZp12f5lZWUqLS0NfxwKhYgQANwA4v427DFjxigjI0ONjY29Pu/3+5WamhqxAQAGvrgH6NixYzp58qSys7PjfSgAQALx/E9wp0+fjriaaW5u1oEDB5Senq709HS9/PLLWrRokYLBoJqamvTcc8/p9ttv15w5c2I6OAAgsXkO0L59+/TAAw+EP/72+zdLlizRhg0bdPDgQf3hD3/QqVOnlJOTo9mzZ+uXv/yl/H5/7KYGACQ8n3POWQ/xt0KhkAKBgGZqvgb7hliPg6tISknxvKZne8Dzmh3jK70fp09vw9k37n/+ac9reh698o9AXM2/TN7sec3cZT/xvCZ55//wvAb933nXrWpVqr29/arf1+decAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADAR81/JjRtHT0eH90WzvK958GHvd1k+MaXvvra65QvvN5QPvFPnec1f/6HL85ovJ2/xvEaSft9+m+c1w//c4nnNec8rMJBwBQQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmOBmpOj3hm/b63nNbdviMIixLx/8r57X9KgnqmNVNNzveU3O0c+jOhZuXFwBAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmuBkpYGDQ3eOiWFXvecVX589FcRwp6/WhUa0DvOAKCABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwwc1IAQP/a21ynxzn3+7/j1GtC+7+LMaTAJfjCggAYIIAAQBMeApQeXm5pk6dqpSUFGVmZmrBggVqaGiI2Ofs2bMqKSnRiBEjdPPNN2vRokVqa2uL6dAAgMTnKUA1NTUqKSlRXV2dPvzwQ3V3d2v27Nnq7OwM77N69Wp98MEH2rp1q2pqanT8+HEtXLgw5oMDABKbpzch7Ny5M+LjTZs2KTMzU/X19ZoxY4ba29v1+9//Xps3b9aDDz4oSdq4caN++MMfqq6uTj/+8Y9jNzkAIKFd1/eA2tvbJUnp6emSpPr6enV3d6uoqCi8z/jx4zVq1CjV1tb2+jm6uroUCoUiNgDAwBd1gHp6erRq1SpNnz5dEyZMkCS1trYqOTlZaWlpEftmZWWptbW1189TXl6uQCAQ3nJzc6MdCQCQQKIOUElJiQ4dOqQtW7Zc1wBlZWVqb28Pb0ePHr2uzwcASAxR/SDqypUrtWPHDu3Zs0cjR44MPx4MBnXu3DmdOnUq4iqora1NwWCw18/l9/vl9/ujGQMAkMA8XQE557Ry5Upt27ZNu3btUl5eXsTzU6ZM0ZAhQ1RVVRV+rKGhQUeOHFFhYWFsJgYADAieroBKSkq0efNmVVZWKiUlJfx9nUAgoGHDhikQCGjZsmUqLS1Venq6UlNT9fTTT6uwsJB3wAEAIngK0IYNGyRJM2fOjHh848aNWrp0qSTpt7/9rZKSkrRo0SJ1dXVpzpw5+t3vfheTYQEAA4enADnnrrnP0KFDVVFRoYqKiqiHAhKJK8z3vOafCqL5omyo5xW+qluiOA7QN7gXHADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAExE9RtRAXznxNSbPK/JG+z9ztY96vG8ZvDZa9/BHrDCFRAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIKbkQLX6WyG9xt+RnNj0fX/5y7Pa0b8l1rPa4C+whUQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCm5EC1+nfLdjdJ8d5q7LI85rbxM1I0X9xBQQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmOBmpMB1+sfmyZ7XPDviX2M/CJBguAICAJggQAAAE54CVF5erqlTpyolJUWZmZlasGCBGhoaIvaZOXOmfD5fxPbkk0/GdGgAQOLzFKCamhqVlJSorq5OH374obq7uzV79mx1dnZG7Ld8+XK1tLSEt3Xr1sV0aABA4vP0JoSdO3dGfLxp0yZlZmaqvr5eM2bMCD8+fPhwBYPB2EwIABiQrut7QO3t7ZKk9PT0iMffeecdZWRkaMKECSorK9OZM2eu+Dm6uroUCoUiNgDAwBf127B7enq0atUqTZ8+XRMmTAg//thjj2n06NHKycnRwYMH9fzzz6uhoUHvv/9+r5+nvLxcL7/8crRjAAASVNQBKikp0aFDh/Txxx9HPL5ixYrwnydOnKjs7GzNmjVLTU1NGjt27GWfp6ysTKWlpeGPQ6GQcnNzox0LAJAgogrQypUrtWPHDu3Zs0cjR4686r4FBQWSpMbGxl4D5Pf75ff7oxkDAJDAPAXIOaenn35a27ZtU3V1tfLy8q655sCBA5Kk7OzsqAYEAAxMngJUUlKizZs3q7KyUikpKWptbZUkBQIBDRs2TE1NTdq8ebMeeughjRgxQgcPHtTq1as1Y8YMTZo0KS5/AQBAYvIUoA0bNki6+MOmf2vjxo1aunSpkpOT9dFHH2n9+vXq7OxUbm6uFi1apBdeeCFmAwMABgbP/wR3Nbm5uaqpqbmugQAANwbuhg1cJ1eVfu2dLvHzkQWe12Ttu+B5DdCfcTNSAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAENyMFrlPW6594XnPode/HGaZPvS8C+jGugAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJjod/eCc85Jks6rW3LGwwAAPDuvbknf/f/8SvpdgDo6OiRJH+ufjScBAFyPjo4OBQKBKz7vc9dKVB/r6enR8ePHlZKSIp/PF/FcKBRSbm6ujh49qtTUVKMJ7XEeLuI8XMR5uIjzcFF/OA/OOXV0dCgnJ0dJSVf+Tk+/uwJKSkrSyJEjr7pPamrqDf0C+xbn4SLOw0Wch4s4DxdZn4erXfl8izchAABMECAAgImECpDf79fatWvl9/utRzHFebiI83AR5+EizsNFiXQe+t2bEAAAN4aEugICAAwcBAgAYIIAAQBMECAAgImECVBFRYVuu+02DR06VAUFBfr000+tR+pzL730knw+X8Q2fvx467Hibs+ePZo3b55ycnLk8/m0ffv2iOedc1qzZo2ys7M1bNgwFRUV6fDhwzbDxtG1zsPSpUsve33MnTvXZtg4KS8v19SpU5WSkqLMzEwtWLBADQ0NEfucPXtWJSUlGjFihG6++WYtWrRIbW1tRhPHx/c5DzNnzrzs9fDkk08aTdy7hAjQe++9p9LSUq1du1afffaZ8vPzNWfOHJ04ccJ6tD539913q6WlJbx9/PHH1iPFXWdnp/Lz81VRUdHr8+vWrdPrr7+uN998U3v37tVNN92kOXPm6OzZs308aXxd6zxI0ty5cyNeH++++24fThh/NTU1KikpUV1dnT788EN1d3dr9uzZ6uzsDO+zevVqffDBB9q6datqamp0/PhxLVy40HDq2Ps+50GSli9fHvF6WLdundHEV+ASwLRp01xJSUn44wsXLricnBxXXl5uOFXfW7t2rcvPz7cew5Qkt23btvDHPT09LhgMuldffTX82KlTp5zf73fvvvuuwYR949Lz4JxzS5YscfPnzzeZx8qJEyecJFdTU+Ocu/jffsiQIW7r1q3hfb744gsnydXW1lqNGXeXngfnnLv//vvdT3/6U7uhvod+fwV07tw51dfXq6ioKPxYUlKSioqKVFtbaziZjcOHDysnJ0djxozR448/riNHjliPZKq5uVmtra0Rr49AIKCCgoIb8vVRXV2tzMxMjRs3Tk899ZROnjxpPVJctbe3S5LS09MlSfX19eru7o54PYwfP16jRo0a0K+HS8/Dt9555x1lZGRowoQJKisr05kzZyzGu6J+dzPSS3399de6cOGCsrKyIh7PysrSl19+aTSVjYKCAm3atEnjxo1TS0uLXn75Zd133306dOiQUlJSrMcz0draKkm9vj6+fe5GMXfuXC1cuFB5eXlqamrSz3/+cxUXF6u2tlaDBg2yHi/menp6tGrVKk2fPl0TJkyQdPH1kJycrLS0tIh9B/LrobfzIEmPPfaYRo8erZycHB08eFDPP/+8Ghoa9P777xtOG6nfBwjfKS4uDv950qRJKigo0OjRo/XHP/5Ry5YtM5wM/cHixYvDf544caImTZqksWPHqrq6WrNmzTKcLD5KSkp06NChG+L7oFdzpfOwYsWK8J8nTpyo7OxszZo1S01NTRo7dmxfj9mrfv9PcBkZGRo0aNBl72Jpa2tTMBg0mqp/SEtL05133qnGxkbrUcx8+xrg9XG5MWPGKCMjY0C+PlauXKkdO3Zo9+7dEb++JRgM6ty5czp16lTE/gP19XCl89CbgoICSepXr4d+H6Dk5GRNmTJFVVVV4cd6enpUVVWlwsJCw8nsnT59Wk1NTcrOzrYexUxeXp6CwWDE6yMUCmnv3r03/Ovj2LFjOnny5IB6fTjntHLlSm3btk27du1SXl5exPNTpkzRkCFDIl4PDQ0NOnLkyIB6PVzrPPTmwIEDktS/Xg/W74L4PrZs2eL8fr/btGmT+/zzz92KFStcWlqaa21ttR6tT/3sZz9z1dXVrrm52f3pT39yRUVFLiMjw504ccJ6tLjq6Ohw+/fvd/v373eS3Guvveb279/vvvrqK+ecc7/+9a9dWlqaq6ysdAcPHnTz5893eXl57ptvvjGePLaudh46OjrcM88842pra11zc7P76KOP3D333OPuuOMOd/bsWevRY+app55ygUDAVVdXu5aWlvB25syZ8D5PPvmkGzVqlNu1a5fbt2+fKywsdIWFhYZTx961zkNjY6N75ZVX3L59+1xzc7OrrKx0Y8aMcTNmzDCePFJCBMg559544w03atQol5yc7KZNm+bq6uqsR+pzjzzyiMvOznbJycnuBz/4gXvkkUdcY2Oj9Vhxt3v3bifpsm3JkiXOuYtvxX7xxRddVlaW8/v9btasWa6hocF26Di42nk4c+aMmz17trv11lvdkCFD3OjRo93y5csH3Bdpvf39JbmNGzeG9/nmm2/cT37yE3fLLbe44cOHu4cffti1tLTYDR0H1zoPR44ccTNmzHDp6enO7/e722+/3T377LOuvb3ddvBL8OsYAAAm+v33gAAAAxMBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYOL/Ad/Oi9QHbTu+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "IMGNO = 12\n",
    "# Uncomment to see raw numerical values.\n",
    "# print(x_test[IMGNO])\n",
    "plt.imshow(x_test[IMGNO].reshape(HEIGHT, WIDTH));\n",
    "print(\"The label for image number\", IMGNO, \"is\", y_test[IMGNO])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the model\n",
    "Let's start with a very simple linear classifier. This was the first method to be tried on MNIST in 1998, and scored an 88% accuracy. Quite ground breaking at the time!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can build our linear classifier using the [tf.keras API](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras), so we don't have to define or initialize our weights and biases. This happens automatically for us in the background. We can also add a softmax layer to transform the logits into probabilities. Finally, we can compile the model using categorical cross entropy in order to strongly penalize high probability predictions that were incorrect.\n",
    "\n",
    "When building more complex models such as DNNs and CNNs our code will be more readable by using the [tf.keras API](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras). Let's get one working so we can test it and use it as a benchmark."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def linear_model():\n",
    "    # TODO: Build a sequential model and compile it.\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Write Input Functions\n",
    "\n",
    "As usual, we need to specify input functions for training and evaluating. We'll scale each pixel value so it's a decimal value between 0 and 1 as a way of normalizing the data.\n",
    "\n",
    "**TODO 1**: Define the scale function below and build the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "BUFFER_SIZE = 5000\n",
    "BATCH_SIZE = 100\n",
    "\n",
    "\n",
    "def scale(image, label):\n",
    "    # TODO\n",
    "\n",
    "\n",
    "def load_dataset(training=True):\n",
    "    \"\"\"Loads MNIST dataset into a tf.data.Dataset\"\"\"\n",
    "    (x_train, y_train), (x_test, y_test) = mnist\n",
    "    x = x_train if training else x_test\n",
    "    y = y_train if training else y_test\n",
    "    # TODO: a) one-hot encode labels, apply `scale` function, and create dataset.\n",
    "    # One-hot encode the classes\n",
    "    if training:\n",
    "         # TODO\n",
    "    return dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-01-03 16:47:20.077558: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test for training passed!\n",
      "Test for eval passed!\n"
     ]
    }
   ],
   "source": [
    "def create_shape_test(training):\n",
    "    dataset = load_dataset(training=training)\n",
    "    data_iter = dataset.__iter__()\n",
    "    (images, labels) = data_iter.get_next()\n",
    "    expected_image_shape = (BATCH_SIZE, HEIGHT, WIDTH)\n",
    "    expected_label_ndim = 2\n",
    "    assert(images.shape == expected_image_shape)\n",
    "    assert(labels.numpy().ndim == expected_label_ndim)\n",
    "    test_name = 'training' if training else 'eval'\n",
    "    print(\"Test for\", test_name, \"passed!\")\n",
    "\n",
    "\n",
    "create_shape_test(True)\n",
    "create_shape_test(False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Time to train the model! The original MNIST linear classifier had an error rate of 12%. Let's use that to sanity check that our model is learning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-01-03 16:48:36.771073: I tensorflow/core/profiler/lib/profiler_session.cc:131] Profiler session initializing.\n",
      "2023-01-03 16:48:36.771126: I tensorflow/core/profiler/lib/profiler_session.cc:146] Profiler session started.\n",
      "2023-01-03 16:48:36.771171: I tensorflow/core/profiler/lib/profiler_session.cc:164] Profiler session tear down.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-01-03 16:48:40.441275: I tensorflow/core/profiler/lib/profiler_session.cc:131] Profiler session initializing.\n",
      "2023-01-03 16:48:40.442652: I tensorflow/core/profiler/lib/profiler_session.cc:146] Profiler session started.\n",
      "2023-01-03 16:48:40.448798: I tensorflow/core/profiler/lib/profiler_session.cc:66] Profiler session collecting data.\n",
      "2023-01-03 16:48:40.453499: I tensorflow/core/profiler/lib/profiler_session.cc:164] Profiler session tear down.\n",
      "2023-01-03 16:48:40.461814: I tensorflow/core/profiler/rpc/client/save_profile.cc:136] Creating directory: mnist_linear/train/plugins/profile/2023_01_03_16_48_40\n",
      "\n",
      "2023-01-03 16:48:40.466562: I tensorflow/core/profiler/rpc/client/save_profile.cc:142] Dumped gzipped tool data for trace.json.gz to mnist_linear/train/plugins/profile/2023_01_03_16_48_40/tensorflow-2-6-20230103-220816.trace.json.gz\n",
      "2023-01-03 16:48:40.472350: I tensorflow/core/profiler/rpc/client/save_profile.cc:136] Creating directory: mnist_linear/train/plugins/profile/2023_01_03_16_48_40\n",
      "\n",
      "2023-01-03 16:48:40.472770: I tensorflow/core/profiler/rpc/client/save_profile.cc:142] Dumped gzipped tool data for memory_profile.json.gz to mnist_linear/train/plugins/profile/2023_01_03_16_48_40/tensorflow-2-6-20230103-220816.memory_profile.json.gz\n",
      "2023-01-03 16:48:40.473138: I tensorflow/core/profiler/rpc/client/capture_profile.cc:251] Creating directory: mnist_linear/train/plugins/profile/2023_01_03_16_48_40\n",
      "Dumped tool data for xplane.pb to mnist_linear/train/plugins/profile/2023_01_03_16_48_40/tensorflow-2-6-20230103-220816.xplane.pb\n",
      "Dumped tool data for overview_page.pb to mnist_linear/train/plugins/profile/2023_01_03_16_48_40/tensorflow-2-6-20230103-220816.overview_page.pb\n",
      "Dumped tool data for input_pipeline.pb to mnist_linear/train/plugins/profile/2023_01_03_16_48_40/tensorflow-2-6-20230103-220816.input_pipeline.pb\n",
      "Dumped tool data for tensorflow_stats.pb to mnist_linear/train/plugins/profile/2023_01_03_16_48_40/tensorflow-2-6-20230103-220816.tensorflow_stats.pb\n",
      "Dumped tool data for kernel_stats.pb to mnist_linear/train/plugins/profile/2023_01_03_16_48_40/tensorflow-2-6-20230103-220816.kernel_stats.pb\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100/100 - 6s - loss: 1.3512 - accuracy: 0.6617 - val_loss: 0.8007 - val_accuracy: 0.8395\n",
      "\n",
      "Epoch 00001: saving model to mnist_linear/\n",
      "Epoch 2/10\n",
      "100/100 - 2s - loss: 0.6893 - accuracy: 0.8444 - val_loss: 0.5582 - val_accuracy: 0.8723\n",
      "\n",
      "Epoch 00002: saving model to mnist_linear/\n",
      "Epoch 3/10\n",
      "100/100 - 1s - loss: 0.5273 - accuracy: 0.8706 - val_loss: 0.4660 - val_accuracy: 0.8861\n",
      "\n",
      "Epoch 00003: saving model to mnist_linear/\n",
      "Epoch 4/10\n",
      "100/100 - 1s - loss: 0.4599 - accuracy: 0.8844 - val_loss: 0.4176 - val_accuracy: 0.8945\n",
      "\n",
      "Epoch 00004: saving model to mnist_linear/\n",
      "Epoch 5/10\n",
      "100/100 - 2s - loss: 0.4290 - accuracy: 0.8864 - val_loss: 0.3860 - val_accuracy: 0.9013\n",
      "\n",
      "Epoch 00005: saving model to mnist_linear/\n",
      "Epoch 6/10\n",
      "100/100 - 1s - loss: 0.3863 - accuracy: 0.8985 - val_loss: 0.3666 - val_accuracy: 0.9047\n",
      "\n",
      "Epoch 00006: saving model to mnist_linear/\n",
      "Epoch 7/10\n",
      "100/100 - 1s - loss: 0.3822 - accuracy: 0.8977 - val_loss: 0.3515 - val_accuracy: 0.9080\n",
      "\n",
      "Epoch 00007: saving model to mnist_linear/\n",
      "Epoch 8/10\n",
      "100/100 - 2s - loss: 0.3536 - accuracy: 0.9041 - val_loss: 0.3417 - val_accuracy: 0.9110\n",
      "\n",
      "Epoch 00008: saving model to mnist_linear/\n",
      "Epoch 9/10\n",
      "100/100 - 1s - loss: 0.3392 - accuracy: 0.9093 - val_loss: 0.3308 - val_accuracy: 0.9122\n",
      "\n",
      "Epoch 00009: saving model to mnist_linear/\n",
      "Epoch 10/10\n",
      "100/100 - 2s - loss: 0.3421 - accuracy: 0.9073 - val_loss: 0.3243 - val_accuracy: 0.9110\n",
      "\n",
      "Epoch 00010: saving model to mnist_linear/\n"
     ]
    }
   ],
   "source": [
    "NUM_EPOCHS = 10\n",
    "STEPS_PER_EPOCH = 100\n",
    "\n",
    "model = linear_model()\n",
    "train_data = load_dataset()\n",
    "validation_data = load_dataset(training=False)\n",
    "\n",
    "OUTDIR = \"mnist_linear/\"\n",
    "checkpoint_callback = ModelCheckpoint(\n",
    "    OUTDIR, save_weights_only=True, verbose=1)\n",
    "tensorboard_callback = TensorBoard(log_dir=OUTDIR)\n",
    "\n",
    "history = model.fit(\n",
    "    # TODO: specify training/eval data, # epochs, steps per epoch.\n",
    "    verbose=2,\n",
    "    callbacks=[checkpoint_callback, tensorboard_callback]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test to beat benchmark accuracy passed!\n",
      "Test model accuracy is improving passed!\n",
      "Test loss is decreasing passed!\n"
     ]
    }
   ],
   "source": [
    "BENCHMARK_ERROR = .12\n",
    "BENCHMARK_ACCURACY = 1 - BENCHMARK_ERROR\n",
    "\n",
    "accuracy = history.history['accuracy']\n",
    "val_accuracy = history.history['val_accuracy']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "    \n",
    "assert(accuracy[-1] > BENCHMARK_ACCURACY)\n",
    "assert(val_accuracy[-1] > BENCHMARK_ACCURACY)\n",
    "print(\"Test to beat benchmark accuracy passed!\")\n",
    "        \n",
    "assert(accuracy[0] < accuracy[1])\n",
    "assert(accuracy[1] < accuracy[-1])\n",
    "assert(val_accuracy[0] < val_accuracy[1])\n",
    "assert(val_accuracy[1] < val_accuracy[-1])\n",
    "print(\"Test model accuracy is improving passed!\")\n",
    "    \n",
    "assert(loss[0] > loss[1])\n",
    "assert(loss[1] > loss[-1])\n",
    "assert(val_loss[0] > val_loss[1])\n",
    "assert(val_loss[1] > val_loss[-1])\n",
    "print(\"Test loss is decreasing passed!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluating Predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Were you able to get an accuracy of over 90%? Not bad for a linear estimator! Let's make some predictions and see if we can find where the model has trouble. Change the range of values below to find incorrect predictions, and plot the corresponding images. What would you have guessed for these images?\n",
    "\n",
    "**TODO 2**: Change the range below to find an incorrect prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "image number: 8\n",
      "the prediction was 6\n",
      "the actual label is 5\n",
      "\n"
     ]
    }
   ],
   "source": [
    "image_numbers = range(0, 10, 1)  # Change me, please.\n",
    "\n",
    "def load_prediction_dataset():\n",
    "    dataset = (x_test[image_numbers], y_test[image_numbers])\n",
    "    dataset = tf.data.Dataset.from_tensor_slices(dataset)\n",
    "    dataset = dataset.map(scale).batch(len(image_numbers))\n",
    "    return dataset\n",
    "\n",
    "predicted_results = model.predict(load_prediction_dataset())\n",
    "for index, prediction in enumerate(predicted_results):\n",
    "    predicted_value = np.argmax(prediction)\n",
    "    actual_value = y_test[image_numbers[index]]\n",
    "    if actual_value != predicted_value:\n",
    "        print(\"image number: \" + str(image_numbers[index]))\n",
    "        print(\"the prediction was \" + str(predicted_value))\n",
    "        print(\"the actual label is \" + str(actual_value))\n",
    "        print(\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAcNElEQVR4nO3dfXBUdb7n8U8HQguadAwh6UQCBlRwRDI7DGSyKINDLiGuLE87F3yYC5aFBQZngPFhM6uiM1MVB6scry4Du7UOjHUFH2YFRq/DLQ0mXMcEJUJRXDWSbJS4kKDUTXcIEkLy2z9Ye2wJMKftzjcJ71fVqSLd55vz49jlm0M3Jz7nnBMAAL0syXoBAICLEwECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmBlsv4Ju6u7t1+PBhpaSkyOfzWS8HAOCRc05tbW3KyclRUtK5r3P6XIAOHz6s3Nxc62UAAL6lpqYmjRw58pzP97kApaSkSJJu0M0arGTj1QAAvDqtTr2t1yP/Pz+XhAVo3bp1euKJJ9Tc3Kz8/Hw988wzmjJlygXnvvprt8FK1mAfAQKAfuf/32H0Qm+jJORDCC+++KJWr16tNWvW6P3331d+fr6Ki4t19OjRRBwOANAPJSRATz75pJYuXao777xT3/nOd7RhwwYNGzZMv//97xNxOABAPxT3AJ06dUq1tbUqKir660GSklRUVKTq6uqz9u/o6FA4HI7aAAADX9wD9MUXX6irq0tZWVlRj2dlZam5ufms/cvLyxUIBCIbn4ADgIuD+T9ELSsrUygUimxNTU3WSwIA9IK4fwouIyNDgwYNUktLS9TjLS0tCgaDZ+3v9/vl9/vjvQwAQB8X9yugIUOGaNKkSaqoqIg81t3drYqKChUWFsb7cACAfioh/w5o9erVWrx4sb7//e9rypQpeuqpp9Te3q4777wzEYcDAPRDCQnQwoUL9fnnn+uRRx5Rc3Ozvvvd72rHjh1nfTABAHDx8jnnnPUivi4cDisQCGi65nAnBADoh067TlVqu0KhkFJTU8+5n/mn4AAAFycCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgYrD1AgD0PYODWZ5nTl2dk4CVxEfyx/83prm6sjGeZ9I+8HmeSf/wpOeZpH/d63mmr+EKCABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwwc1IgX4idMcPPM8cu9n7TS4l6b/+hx2eZ/4h9fWYjtUbng2NimlufspWzzOX//iSmI7l1S1XTOqV4yQSV0AAABMECABgIu4BevTRR+Xz+aK28ePHx/swAIB+LiHvAV133XV68803/3qQwbzVBACIlpAyDB48WMFgMBHfGgAwQCTkPaCDBw8qJydHY8aM0e23365Dhw6dc9+Ojg6Fw+GoDQAw8MU9QAUFBdq0aZN27Nih9evXq7GxUTfeeKPa2tp63L+8vFyBQCCy5ebmxntJAIA+KO4BKikp0Y9//GNNnDhRxcXFev3119Xa2qqXXnqpx/3LysoUCoUiW1NTU7yXBADogxL+6YC0tDRdc801qq+v7/F5v98vv9+f6GUAAPqYhP87oOPHj6uhoUHZ2dmJPhQAoB+Je4Duu+8+VVVV6ZNPPtE777yjefPmadCgQbr11lvjfSgAQD8W97+C++yzz3Trrbfq2LFjGjFihG644QbV1NRoxIgR8T4UAKAfi3uAXnjhhXh/S6DXJOVf63nmo3sv9TzzrzOf8jwzYtB7nmeSuNuWJOmuwLn/Kcj59c6NRS9WvDoBACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMJ/4F0QH/SnpfieebjkvUxHGloDDOQpA2tYzzPPP/p5ASsxFZAPf+Qz/6EKyAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCY4G7YiNngkVd4nvnwwZGeZ7Le8XmeSd1S43lGkpI6nOeZjztPeZ5pOp3meSZ3cKvnmSUHFnuekaR//3C455ms97yfu7R3mjzPuOPHPc8EWvv/naMHIq6AAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAAT3IwUGpQWiGluyj83ep7ZlvEnzzNT96zwPBMr/5/f8zxz/39a4nmm69/qPM8MuvZqzzPpdQ2eZyQpvfvjmOa8Ot0rR0FfxRUQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCm5EOMEmXXOJ5puOPsd2M9BcZOz3PjHvlHs8z47f+m+eZLs8TsYvlxqIxHefDg71yHKC3cAUEADBBgAAAJjwHaNeuXZo9e7ZycnLk8/m0bdu2qOedc3rkkUeUnZ2toUOHqqioSAcP8lcHAIBongPU3t6u/Px8rVu3rsfn165dq6efflobNmzQ7t27demll6q4uFgnT5781osFAAwcnj+EUFJSopKSkh6fc87pqaee0kMPPaQ5c+ZIkp577jllZWVp27ZtWrRo0bdbLQBgwIjre0CNjY1qbm5WUVFR5LFAIKCCggJVV1f3ONPR0aFwOBy1AQAGvrgGqLm5WZKUlZUV9XhWVlbkuW8qLy9XIBCIbLm5ufFcEgCgjzL/FFxZWZlCoVBka2pqsl4SAKAXxDVAwWBQktTS0hL1eEtLS+S5b/L7/UpNTY3aAAADX1wDlJeXp2AwqIqKishj4XBYu3fvVmFhYTwPBQDo5zx/Cu748eOqr6+PfN3Y2Kh9+/YpPT1do0aN0sqVK/XrX/9aV199tfLy8vTwww8rJydHc+fOjee6AQD9nOcA7dmzRzfddFPk69WrV0uSFi9erE2bNumBBx5Qe3u77r77brW2tuqGG27Qjh07dEkM9ygDAAxcPuecs17E14XDYQUCAU3XHA32JVsvx9Sgyy/3PPPRr67xPFM373eeZySptsP7zC8LZnme6fr8c+8HAmDmtOtUpbYrFAqd931980/BAQAuTgQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDh+ccxoPccvuNazzN1857xPPOndu933ZakZ2/5O88zXZ83xHQsAAMPV0AAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAluRtqHtRV82SvH+cfGGTHNDf2YG4sCiB1XQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACW5G2odtmfo/Y5jy/meKP37nn2I4jlT45M89z+T96ZTnmUGV73ueAdD3cQUEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJjgZqR92BR/sueZTtfleebypEs8z0jSRwvXeZ7p/Hvv65tQsczzTOC92H5Px0c6zzOp/8f7cTL2t3sfisEXEy+NaS6r8qjnma6PG2I6Fi5eXAEBAEwQIACACc8B2rVrl2bPnq2cnBz5fD5t27Yt6vklS5bI5/NFbbNmzYrXegEAA4TnALW3tys/P1/r1p377/9nzZqlI0eORLYtW7Z8q0UCAAYezx9CKCkpUUlJyXn38fv9CgaDMS8KADDwJeQ9oMrKSmVmZmrcuHFavny5jh07ds59Ozo6FA6HozYAwMAX9wDNmjVLzz33nCoqKvSb3/xGVVVVKikpUVdXzx+/LS8vVyAQiGy5ubnxXhIAoA+K+78DWrRoUeTX119/vSZOnKixY8eqsrJSM2bMOGv/srIyrV69OvJ1OBwmQgBwEUj4x7DHjBmjjIwM1dfX9/i83+9Xampq1AYAGPgSHqDPPvtMx44dU3Z2dqIPBQDoRzz/Fdzx48ejrmYaGxu1b98+paenKz09XY899pgWLFigYDCohoYGPfDAA7rqqqtUXFwc14UDAPo3zwHas2ePbrrppsjXX71/s3jxYq1fv1779+/XH/7wB7W2tionJ0czZ87Ur371K/n9/vitGgDQ7/mcc97vvphA4XBYgUBA0zVHg33eb8Y5kHz8PyZ7n7llQwJWgovNux0+zzMrP1h04Z2+If2Wjz3PoO877TpVqe0KhULnfV+fe8EBAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABHfD7sN8g73/xPRT0/M9z/zDf3/V84wkDUvq8Dxzy7DPPc8k+wZ5nkHv61a355nrNv/U88zY+6s9z6B3cTdsAECfRoAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCY8H63S/Qad/q055nkN2s9z2wZn+N5JlZP/5dFnme6kn2eZ/7jfe96npGkx4PvxTQHKSmGP8+OzD+SgJWgv+AKCABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwwc1I0asu/ePuXjnOq/mFMc09/hPvNyM94U55npm0a7nnmdH/a5DnmS9+esLzjCTtmfxPMc0BXnAFBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCY4GakGJBG/UtHbIM/8T4yzDfE88yHP3zW88xPRv+d55nXr/wXzzNn9M6fTQ81p3ueuVqfxH8hMMEVEADABAECAJjwFKDy8nJNnjxZKSkpyszM1Ny5c1VXVxe1z8mTJ1VaWqrhw4frsssu04IFC9TS0hLXRQMA+j9PAaqqqlJpaalqamr0xhtvqLOzUzNnzlR7e3tkn1WrVunVV1/Vyy+/rKqqKh0+fFjz58+P+8IBAP2bpw8h7NixI+rrTZs2KTMzU7W1tZo2bZpCoZCeffZZbd68WT/60Y8kSRs3btS1116rmpoa/eAHP4jfygEA/dq3eg8oFApJktLTz3ySpba2Vp2dnSoqKorsM378eI0aNUrV1dU9fo+Ojg6Fw+GoDQAw8MUcoO7ubq1cuVJTp07VhAkTJEnNzc0aMmSI0tLSovbNyspSc3Nzj9+nvLxcgUAgsuXm5sa6JABAPxJzgEpLS3XgwAG98MIL32oBZWVlCoVCka2pqelbfT8AQP8Q0z9EXbFihV577TXt2rVLI0eOjDweDAZ16tQptba2Rl0FtbS0KBgM9vi9/H6//H5/LMsAAPRjnq6AnHNasWKFtm7dqp07dyovLy/q+UmTJik5OVkVFRWRx+rq6nTo0CEVFhbGZ8UAgAHB0xVQaWmpNm/erO3btyslJSXyvk4gENDQoUMVCAR01113afXq1UpPT1dqaqruvfdeFRYW8gk4AEAUTwFav369JGn69OlRj2/cuFFLliyRJP32t79VUlKSFixYoI6ODhUXF+t3v/tdXBYLABg4fM45Z72IrwuHwwoEApquORrsS7ZeDvqppJSUmOaObs72PFPzvS0xHasv63Cdnmdu+WCR55lhf//vnme6WkOeZ9C7TrtOVWq7QqGQUlNTz7kf94IDAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACAiZh+IirQ13W3tcU0F7z3cs8zs3//nz3P/OLKf/Y8U+jv8jzzv49neJ6RpP/2+kLPM1etqvE84/13hIGEKyAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQ3IwW+5vQnh7wP/cj7yE9/eo/nmbbJX3qeGf/QF55nJOmqT73fWBTwiisgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAENyMFDGQ9/Y73mRiOczqGGaC3cAUEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATHgKUHl5uSZPnqyUlBRlZmZq7ty5qquri9pn+vTp8vl8UduyZcviumgAQP/nKUBVVVUqLS1VTU2N3njjDXV2dmrmzJlqb2+P2m/p0qU6cuRIZFu7dm1cFw0A6P88/UTUHTt2RH29adMmZWZmqra2VtOmTYs8PmzYMAWDwfisEAAwIH2r94BCoZAkKT09Perx559/XhkZGZowYYLKysp04sSJc36Pjo4OhcPhqA0AMPB5ugL6uu7ubq1cuVJTp07VhAkTIo/fdtttGj16tHJycrR//349+OCDqqur0yuvvNLj9ykvL9djjz0W6zIAAP2UzznnYhlcvny5/vznP+vtt9/WyJEjz7nfzp07NWPGDNXX12vs2LFnPd/R0aGOjo7I1+FwWLm5uZquORrsS45laQAAQ6ddpyq1XaFQSKmpqefcL6YroBUrVui1117Trl27zhsfSSooKJCkcwbI7/fL7/fHsgwAQD/mKUDOOd17773aunWrKisrlZeXd8GZffv2SZKys7NjWiAAYGDyFKDS0lJt3rxZ27dvV0pKipqbmyVJgUBAQ4cOVUNDgzZv3qybb75Zw4cP1/79+7Vq1SpNmzZNEydOTMhvAADQP3l6D8jn8/X4+MaNG7VkyRI1NTXpjjvu0IEDB9Te3q7c3FzNmzdPDz300Hn/HvDrwuGwAoEA7wEBQD+VkPeALtSq3NxcVVVVefmWAICLFPeCAwCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYGGy9gG9yzkmSTqtTcsaLAQB4dlqdkv76//Nz6XMBamtrkyS9rdeNVwIA+Dba2toUCATO+bzPXShRvay7u1uHDx9WSkqKfD5f1HPhcFi5ublqampSamqq0QrtcR7O4DycwXk4g/NwRl84D845tbW1KScnR0lJ536np89dASUlJWnkyJHn3Sc1NfWifoF9hfNwBufhDM7DGZyHM6zPw/mufL7ChxAAACYIEADARL8KkN/v15o1a+T3+62XYorzcAbn4QzOwxmchzP603nocx9CAABcHPrVFRAAYOAgQAAAEwQIAGCCAAEATPSbAK1bt05XXnmlLrnkEhUUFOjdd9+1XlKve/TRR+Xz+aK28ePHWy8r4Xbt2qXZs2crJydHPp9P27Zti3reOadHHnlE2dnZGjp0qIqKinTw4EGbxSbQhc7DkiVLznp9zJo1y2axCVJeXq7JkycrJSVFmZmZmjt3rurq6qL2OXnypEpLSzV8+HBddtllWrBggVpaWoxWnBh/y3mYPn36Wa+HZcuWGa24Z/0iQC+++KJWr16tNWvW6P3331d+fr6Ki4t19OhR66X1uuuuu05HjhyJbG+//bb1khKuvb1d+fn5WrduXY/Pr127Vk8//bQ2bNig3bt369JLL1VxcbFOnjzZyytNrAudB0maNWtW1Otjy5YtvbjCxKuqqlJpaalqamr0xhtvqLOzUzNnzlR7e3tkn1WrVunVV1/Vyy+/rKqqKh0+fFjz5883XHX8/S3nQZKWLl0a9XpYu3at0YrPwfUDU6ZMcaWlpZGvu7q6XE5OjisvLzdcVe9bs2aNy8/Pt16GKUlu69atka+7u7tdMBh0TzzxROSx1tZW5/f73ZYtWwxW2Du+eR6cc27x4sVuzpw5JuuxcvToUSfJVVVVOefO/LdPTk52L7/8cmSfDz/80Ely1dXVVstMuG+eB+ec++EPf+h+9rOf2S3qb9Dnr4BOnTql2tpaFRUVRR5LSkpSUVGRqqurDVdm4+DBg8rJydGYMWN0++2369ChQ9ZLMtXY2Kjm5uao10cgEFBBQcFF+fqorKxUZmamxo0bp+XLl+vYsWPWS0qoUCgkSUpPT5ck1dbWqrOzM+r1MH78eI0aNWpAvx6+eR6+8vzzzysjI0MTJkxQWVmZTpw4YbG8c+pzNyP9pi+++EJdXV3KysqKejwrK0sfffSR0apsFBQUaNOmTRo3bpyOHDmixx57TDfeeKMOHDiglJQU6+WZaG5ulqQeXx9fPXexmDVrlubPn6+8vDw1NDToF7/4hUpKSlRdXa1BgwZZLy/uuru7tXLlSk2dOlUTJkyQdOb1MGTIEKWlpUXtO5BfDz2dB0m67bbbNHr0aOXk5Gj//v168MEHVVdXp1deecVwtdH6fIDwVyUlJZFfT5w4UQUFBRo9erReeukl3XXXXYYrQ1+waNGiyK+vv/56TZw4UWPHjlVlZaVmzJhhuLLEKC0t1YEDBy6K90HP51zn4e677478+vrrr1d2drZmzJihhoYGjR07treX2aM+/1dwGRkZGjRo0FmfYmlpaVEwGDRaVd+Qlpama665RvX19dZLMfPVa4DXx9nGjBmjjIyMAfn6WLFihV577TW99dZbUT++JRgM6tSpU2ptbY3af6C+Hs51HnpSUFAgSX3q9dDnAzRkyBBNmjRJFRUVkce6u7tVUVGhwsJCw5XZO378uBoaGpSdnW29FDN5eXkKBoNRr49wOKzdu3df9K+Pzz77TMeOHRtQrw/nnFasWKGtW7dq586dysvLi3p+0qRJSk5Ojno91NXV6dChQwPq9XCh89CTffv2SVLfej1Yfwrib/HCCy84v9/vNm3a5D744AN39913u7S0NNfc3Gy9tF7185//3FVWVrrGxkb3l7/8xRUVFbmMjAx39OhR66UlVFtbm9u7d6/bu3evk+SefPJJt3fvXvfpp58655x7/PHHXVpamtu+fbvbv3+/mzNnjsvLy3Nffvml8crj63znoa2tzd13332uurraNTY2ujfffNN973vfc1dffbU7efKk9dLjZvny5S4QCLjKykp35MiRyHbixInIPsuWLXOjRo1yO3fudHv27HGFhYWusLDQcNXxd6HzUF9f7375y1+6PXv2uMbGRrd9+3Y3ZswYN23aNOOVR+sXAXLOuWeeecaNGjXKDRkyxE2ZMsXV1NRYL6nXLVy40GVnZ7shQ4a4K664wi1cuNDV19dbLyvh3nrrLSfprG3x4sXOuTMfxX744YddVlaW8/v9bsaMGa6urs520QlwvvNw4sQJN3PmTDdixAiXnJzsRo8e7ZYuXTrg/pDW0+9fktu4cWNkny+//NLdc8897vLLL3fDhg1z8+bNc0eOHLFbdAJc6DwcOnTITZs2zaWnpzu/3++uuuoqd//997tQKGS78G/gxzEAAEz0+feAAAADEwECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABg4v8BaZna//jw0ukAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "bad_image_number = 8\n",
    "plt.imshow(x_test[bad_image_number].reshape(HEIGHT, WIDTH));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's understandable why the poor computer would have some trouble. Some of these images are difficult for even humans to read. In fact, we can see what the computer thinks each digit looks like.\n",
    "\n",
    "Each of the 10 neurons in the dense layer of our model has 785 weights feeding into it. That's 1 weight for every pixel in the image + 1 for a bias term. These weights are flattened feeding into the model, but we can reshape them back into the original image dimensions to see what the computer sees.\n",
    "\n",
    "**TODO 3**: Reshape the layer weights to be the shape of an input image and plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f8fa09f0d90>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAp9klEQVR4nO3de3Bc9Znm8adbl9a9ZUnWDcu25CvgS4LBjmNwnLHGl+ywELxZIKlaw2RhYeRswJPLOptAyGRGCdlhqKQ8ULOb4KQqQGA3wMBkPAET20NiO2ODYwxGWEZGMtbNstUttW4t9dk/XNZEYEO/B8k/SXw/VV1lSefR+fXR6X7U7u5XAc/zPAEAcJEFXS8AAPDRRAEBAJyggAAATlBAAAAnKCAAgBMUEADACQoIAOAEBQQAcCLV9QLeLZFI6OTJk8rNzVUgEHC9HACAked56urqUnl5uYLBCz/OGXcFdPLkSVVUVLheBgDgQ2pqatK0adMu+PVxV0C5ubmSpFX/91alZqUnnTvdk2XeV3tHrjkjSctmHTdn9r4+y5xJidp/PJcsbDZnIr2Z5owk9falmTPlBRFzprGlwJwJtmSYM5IUHLRnKpc1mjOdffZjfvrwVHPGm9ZrzkhSSUHUnGmPZpszg832TCIjYc58cuGb5owkdfTZ13f0kP0X6ETYx4kX9/c/REWX2G+DHcenmLZP9PXpnW/+9fD9+YWMWQFt3bpVP/jBD9TS0qLFixfrRz/6kZYuXfqBuXP/7Zaala607OQLKCUQMq8x2OPvTsqyruF9Zdr3FRyw/3hSs+3Hwc+xk6Rg0H4cUrP77PvJ8nHsMi5eAfk65ik+zlcf18nL8jfqMTW735xJGbSvL+Hn55RpLyA/t1lJSr1IPydl+jnx/BVQSpaP26CP+y9JH/g0ypi8COEXv/iFNm/erHvvvVcvv/yyFi9erLVr16qtrW0sdgcAmIDGpIAeeOAB3Xbbbbr11lt12WWX6eGHH1ZWVpZ+8pOfjMXuAAAT0KgX0MDAgA4cOKDq6up/30kwqOrqau3Zs+c92/f39ysajY64AAAmv1EvoFOnTmloaEglJSUjPl9SUqKWlpb3bF9bW6twODx84RVwAPDR4PyNqFu2bFEkEhm+NDU1uV4SAOAiGPVXwRUVFSklJUWtra0jPt/a2qrS0tL3bB8KhRQK+XsVFgBg4hr1R0Dp6elasmSJduzYMfy5RCKhHTt2aPny5aO9OwDABDUm7wPavHmzNm7cqCuvvFJLly7Vgw8+qFgspltvvXUsdgcAmIDGpIBuvPFGtbe365577lFLS4s+9rGPafv27e95YQIA4KMr4Hmev7dKj5FoNKpwOKyqn25RiuEd8CumN5j31d6fY85I0uFDM8yZsjnt5kzra8XmzFBB3JwJpPg7BT420/6CkZPdYXOm/c0ic2b65faRRJL0dnOhOeP1+Pg9Lt3+bv6U0/b9ZFR1mTOS1NtoH1MVmtZtzqSlDpkzg0P2Zw4G38gzZyQpdb79bSFTsu3jj1r/YP/lfLDIfluXpIw8+5SLoSHb1IVET58abv0bRSIR5eVd+Ng7fxUcAOCjiQICADhBAQEAnKCAAABOUEAAACcoIACAExQQAMAJCggA4AQFBABwggICADhBAQEAnKCAAABOjMk07NGQlRFXSkby/fj6Gfswv1nhDnNGkjLKYubMlAz7gML2uG0AoCQNDdp/p5gxrc2ckaSj/zjHV84qpcg+LLX3p2W+9jUl237M02L29cWzUsyZ/PoBc+bUIvvwV0nK77Vfp6Fj9oGfiTRzRINT7GtL6bf/XCWppzPTnOlrsg9yTe/xsb52HwdPUn+f/dwLhG3nXmIwuX3wCAgA4AQFBABwggICADhBAQEAnKCAAABOUEAAACcoIACAExQQAMAJCggA4AQFBABwggICADhBAQEAnKCAAABOjNtp2NGGfAUzMpLePn/2afM+Xmm+xJyRpKE6+7Tb40EfE5PDQ+ZMaof9R9rzr+XmjCR5PgZO9xUnzJn0M/bfk/rz/E0/jsyzr8/LsP+cMk7aJxkHEunmTN9U+3knSakx+/HLOGXfV3qXOaK0bnsmtyluD0kaet0+Obr5k/b9pPTbM33T/V2nQI/9OgVPJn9fLEnqS/L7mlcCAMAooIAAAE5QQAAAJyggAIATFBAAwAkKCADgBAUEAHCCAgIAOEEBAQCcoIAAAE5QQAAAJyggAIAT43YYaV5lp1KyQklvH+nKMu8je589I0mX/qc3zZk/7JljzmRGfAzU9BHp9jeTVUOZ9uGTXrF96uLggHEQoqSUfn/DSFN67bnBPPsA03iuPXP6avvwydz8HnNGkroimeZMX7F9WKqXb79OqS32/cRz7MNfJal7cZJTNf9I1hH7+ZoWNUcUDNmH4EpS3gH78Ysstd1uE70DSW3HIyAAgBMUEADACQoIAOAEBQQAcIICAgA4QQEBAJyggAAATlBAAAAnKCAAgBMUEADACQoIAOAEBQQAcGLcDiPtfCdPwczkh/pVzGo376P1k/aBkJLU0FlgzgR8zA3MOWEf9nnmUvt+co/bM5I0OGAf3Bk6YB9y2VtsPw7xXHtGkuYse9ucCQbs+6peesSc6RlKfjjvOU82fNyckaRNS3aaM4+/vcSc6TiTY87Ei+0DTHMu6zJnJCn1NfttvT/ffj5kdJgjKn3GPlRUkjrtc5Hl9aTYtu9NbnseAQEAnKCAAABOjHoBffvb31YgEBhxmT9//mjvBgAwwY3Jc0CXX365XnjhhX/fSeq4faoJAODImDRDamqqSktLx+JbAwAmiTF5Dujo0aMqLy9XVVWVvvCFL6ixsfGC2/b39ysajY64AAAmv1EvoGXLlmnbtm3avn27HnroITU0NOiaa65RV9f5XwZZW1urcDg8fKmoqBjtJQEAxqFRL6D169frc5/7nBYtWqS1a9fqV7/6lTo7O/XEE0+cd/stW7YoEokMX5qamkZ7SQCAcWjMXx2Qn5+vuXPnqr6+/rxfD4VCCoXsb7ADAExsY/4+oO7ubh07dkxlZWVjvSsAwAQy6gX0la98Rbt27dLx48f1u9/9Tp/97GeVkpKim2++ebR3BQCYwEb9v+BOnDihm2++WR0dHZo6daquvvpq7d27V1OnTh3tXQEAJrBRL6DHH398VL5PMB5UMCX5B2gnO8LmfYRezTJnJOlMuX2gppdvn0Ya+UyfOTPYbR9QGKxPM2ckf0NCvVR7JnTK/kA9ddkZc0aSjtRNM2fKZ54yZ97ssb9P7hO5538e9f1sqDxozkhSmo/puffOe9ac+eWpK82Zjv5scyY16GMasKQDhXn2UNB+jvd32G+3RS93mzOSpECuOZL4eK9p+6Ge5O67mAUHAHCCAgIAOEEBAQCcoIAAAE5QQAAAJyggAIATFBAAwAkKCADgBAUEAHCCAgIAOEEBAQCcoIAAAE6M+R+k8yuROyhlDia9feCU/Y/aeT6vfXAgYM5kNqaYM4H6HHPGm2ofhNhxhb9BjVMO2a/TQL79d57BK87/59zfj31lZ12z+A1z5qWjs82Z9BT7Me8etA+s3Fz6vDkjSdtOrzBnrsp8y5yZm91qzjzR/nFzpj/ub+BuVmGPOdNzyj7kuPBw3JxJecc+BFeS7KNIpTbj8UvEk7sF8ggIAOAEBQQAcIICAgA4QQEBAJyggAAATlBAAAAnKCAAgBMUEADACQoIAOAEBQQAcIICAgA4QQEBAJyggAAATozbadjBSKqC/ckvL63L3qWez5HJqd32fWW12KdUt68eMGfS37ZPBc+r83caROYmzJlETvITzs+ZkmE/DpGIfSKxJP1r+zxzJtvHxGQ/rs6vN2ceaPlTX/v6X9P+2Zz5nyfXmDOr8183Z9JT7ZPETzeHzRlJqphhnzh9XdWr5swzx682Z7Jett+nSFIiZL+9BxsybYG+5P5iAI+AAABOUEAAACcoIACAExQQAMAJCggA4AQFBABwggICADhBAQEAnKCAAABOUEAAACcoIACAExQQAMCJcTuMNDUWUHAouYF2ktRfFjfvIzDgr39Dp+xTTGPlyV+Xc3JetQ8W7Sm3DwhNj16830OC3fZjd6Ylz74jf3Matebjh82ZhGc/fpF4hjnz/QNrzZnZZe3mjCTd/tYN5szfzvx/5syOnrnmzJx8+3VaV37EnJGk352qMmf2dcw0Z3rm9pszGrIPZZWklB77feVY4REQAMAJCggA4AQFBABwggICADhBAQEAnKCAAABOUEAAACcoIACAExQQAMAJCggA4AQFBABwggICADgxboeRxvMTCmYmP1gzqyHNvI/BHH8TKwfy7AM/04L2YaSDWfb1ZZ+w/07RvaTXnJGk1DT7MMR4e6Y5UzLtjDnTfjrXnJGkI2dKzZm8UJ850xbLMWeKfm0fYPrmp0vMGUmqKD9tzvy2d6Y5k58SM2fmZ7eaMyVpEXNGkv6zj+PwX/LeMWe+OmWZOfNm4UxzRpJ6y7LNmQHjsOdEb3Lb8wgIAOAEBQQAcMJcQLt379a1116r8vJyBQIBPf300yO+7nme7rnnHpWVlSkzM1PV1dU6evToaK0XADBJmAsoFotp8eLF2rp163m/fv/99+uHP/yhHn74Ye3bt0/Z2dlau3at+vrs/08OAJi8zC9CWL9+vdavX3/er3mepwcffFDf/OY3dd1110mSfvazn6mkpERPP/20brrppg+3WgDApDGqzwE1NDSopaVF1dXVw58Lh8NatmyZ9uzZc95Mf3+/otHoiAsAYPIb1QJqaWmRJJWUjHzpZ0lJyfDX3q22tlbhcHj4UlFRMZpLAgCMU85fBbdlyxZFIpHhS1NTk+slAQAuglEtoNLSs2/ia20d+Uax1tbW4a+9WygUUl5e3ogLAGDyG9UCqqysVGlpqXbs2DH8uWg0qn379mn58uWjuSsAwARnfhVcd3e36uvrhz9uaGjQwYMHVVBQoOnTp+uuu+7Sd7/7Xc2ZM0eVlZX61re+pfLycl1//fWjuW4AwARnLqD9+/fr05/+9PDHmzdvliRt3LhR27Zt09e+9jXFYjHdfvvt6uzs1NVXX63t27crI8M+xwoAMHkFPM/zN5FzjESjUYXDYc3+6t8oJZR8afXO7jfvK+dIyJyRpJ5y+zDSrJM+hoRWDZoz5TNPmTNtp/0975aTbX9zcVfM/ovI1Cld5sy68iPmjCT9unm+OXPqgH3gZ2rMPpx26h9sAyElKa3bfg5JUmSm/ecUmWvfz5+t22fOXBLqtO/Ipxnp9ttTbtA+3LcpXmjO/OCJG8wZScp+x36X373aNjQ20dOnt275G0Uikfd9Xt/5q+AAAB9NFBAAwAkKCADgBAUEAHCCAgIAOEEBAQCcoIAAAE5QQAAAJyggAIATFBAAwAkKCADgBAUEAHCCAgIAOGH+cwwXy2Cup0RG8lNbU06nmfcx5G8YtjJb7L2dHrFPoM0otE/VLczsMWfa5G8adnwoxZwZ6rOfcp+reMWc2XrwU+aMJOXl2o958QH7dPTAkP18yP7DSXMmtrDcnJGk3BMD5kx0Vro582+nZpgzoWL7hO/OwSxzRpIa+wvMme+WvGTO/I8zV5gzFVc3mTOSVP/W+f869fvJTDGe40luzyMgAIATFBAAwAkKCADgBAUEAHCCAgIAOEEBAQCcoIAAAE5QQAAAJyggAIATFBAAwAkKCADgBAUEAHBi3A4jLby8XSnZyU8LnZPfbt7HSwcuNWckqWyXj1AgYM9k9Jsjx89MMWeWzGg0ZySpKD1mzvxzz2XmjJ+BkOE8+1BWSYodsu+r8Iz955TedMacGSrJN2eyX/H3s+2+0j4kNJCwn+Md3fYhoT1F9qGnL7dPM2ck6W/nP2HOvB63D+n9sykHzZlfHfmCOSNJRb+z3+2fXpRj2j6R5NBhHgEBAJyggAAATlBAAAAnKCAAgBMUEADACQoIAOAEBQQAcIICAgA4QQEBAJyggAAATlBAAAAnKCAAgBPjdhhpbnq/Ug0zB/e9eLl5H1lRHwNCJaX2DZozjZ/xzJkvV/3enHn87SvNmRlZp80ZSbo884Q50zIj15xp67NnurozzRlJyn/LnhnIs9+M0rIz7DsK2n9fPPWnVfb9SAp1DZkzwQH7fuID9mP3wtvzzJmqwg5zRpJ+0rbSnPnx9JfMmTfj9tugF/f3+OH0Qvt9Uei0bV9D/cltzyMgAIATFBAAwAkKCADgBAUEAHCCAgIAOEEBAQCcoIAAAE5QQAAAJyggAIATFBAAwAkKCADgBAUEAHBi3A4jPX5wmoIZyQ9sLLuyxbyPjp1l5owkdc6yH7aSGW3mzMOvXWPOLJ9+3JxJC9gHT0pS40CROVMYipkzK/KOmjP7d883ZySp8NVucyZYbx/KOnTmjDmTUlRozqRWzjFnJCk63X6Oxxfaf7Z+/Ld59mGfPQnDZOM/kpPSZ85s7wmZM1Vp9mOXld9rzkhSX8Q+3Hcw0zbANBFIbnseAQEAnKCAAABOmAto9+7duvbaa1VeXq5AIKCnn356xNdvueUWBQKBEZd169aN1noBAJOEuYBisZgWL16srVu3XnCbdevWqbm5efjy2GOPfahFAgAmH/MzjevXr9f69evfd5tQKKTS0lLfiwIATH5j8hzQzp07VVxcrHnz5unOO+9UR8eF/xxuf3+/otHoiAsAYPIb9QJat26dfvazn2nHjh36/ve/r127dmn9+vUaGjr/S31ra2sVDoeHLxUVFaO9JADAODTq7wO66aabhv+9cOFCLVq0SLNmzdLOnTu1evXq92y/ZcsWbd68efjjaDRKCQHAR8CYvwy7qqpKRUVFqq+vP+/XQ6GQ8vLyRlwAAJPfmBfQiRMn1NHRobIyf1MHAACTk/m/4Lq7u0c8mmloaNDBgwdVUFCggoIC3XfffdqwYYNKS0t17Ngxfe1rX9Ps2bO1du3aUV04AGBiMxfQ/v379elPf3r443PP32zcuFEPPfSQDh06pJ/+9Kfq7OxUeXm51qxZo7/6q79SKGSfjwQAmLzMBbRq1Sp53oUHzf3Lv/zLh1rQOYlUT0q1DcCziuf5+/5TjibMmdZDU82ZlCr7YMzXOuzvv5pV0W7OSFLXUPLDYs/5XtkOc+aGN242Z/xqW5JjzpSdsj9vmZqdZc7EFpabM35FLhu0h/rsr2mqvuwNc+aB360xZ6oqW80ZSVpdXGfO/KpnkTmz88RscyZ9R9ickaT+Svv9Xsoc431RT3JDXJkFBwBwggICADhBAQEAnKCAAABOUEAAACcoIACAExQQAMAJCggA4AQFBABwggICADhBAQEAnKCAAABOUEAAACdG/U9yj5biee1KzU7+Tzi0/t4+BVop9ogkdZfbg8FB+wTaG+e9bM5MSY2ZM082XWHOSNKfz/idOfN/IgvNmXB6cpN1/9jbefaJ5ZKkgP13ssgV9nMvLWZf32CWfW2BIX8T3zOa7XcNuUvPmDMd/fap4Os/9qo5UxcpNmck6cygfX3zslrMmX/qWmDO+P7b0QH7OTFr6inT9vHYgI4lsR2PgAAATlBAAAAnKCAAgBMUEADACQoIAOAEBQQAcIICAgA4QQEBAJyggAAATlBAAAAnKCAAgBMUEADAiXE7jLTjD8UKZmQkvf3ca46b9/Ha0WnmjCR5rWnmzEBZ3Jw50TvFnEnLGjJnZoVtgwbPiXv2oaxfLUhmROFI7/Tbj8Nbb80yZyRpwMeEx7SYj9/jAvZI25X2/QT7fexIUsXVTeZMT9x+uzj4iv3nFCjsN2cSXfa1SVK8yn6Ob++51JxJ9Nv3k9rrb9BsdpP9PHq9uMy0faI3uQHCPAICADhBAQEAnKCAAABOUEAAACcoIACAExQQAMAJCggA4AQFBABwggICADhBAQEAnKCAAABOUEAAACfG7TDSoQxPXkbyw/be2D/DvpMpg/aMpLxG+8DPrrn2YYMvvnyZOVO95DVz5ni00Jzxm8tP2WnOfLnInjn+H/1dp9f2VpkzQyEfAz+D9vMhUdFrzmRkJzcU8t2Csg+6bG7NN2dSi+3XKR5NN2dWffyIOSNJvUP2IaZtkRxzJqXTflecsB8GSVLAxwzTlOaQbR99ye2ER0AAACcoIACAExQQAMAJCggA4AQFBABwggICADhBAQEAnKCAAABOUEAAACcoIACAExQQAMAJCggA4MS4HUb6yaVHlJ6T/LS9fzs53byP+Ot55owknZlnHz4ZzLcPXawsPWXOvPDqpeZM1cw2c0aSstMGzJknWq80Z26afcacGfT8/W41Z+nb5syR42X2HXn2c2hm8Wlz5vixEnNGkrpacs2Z/LKoORPrtU/UDMTtP9v9zRXmjCQNvBY2Z4YMQ5TPST9jv07d03xMFZU0UG6/3c6osN0XDcb6dTyJ7XgEBABwggICADhhKqDa2lpdddVVys3NVXFxsa6//nrV1dWN2Kavr081NTUqLCxUTk6ONmzYoNbW1lFdNABg4jMV0K5du1RTU6O9e/fq+eefVzwe15o1axSLxYa3ufvuu/Xss8/qySef1K5du3Ty5EndcMMNo75wAMDEZnoRwvbt20d8vG3bNhUXF+vAgQNauXKlIpGIfvzjH+vRRx/Vn/zJn0iSHnnkEV166aXau3evPvGJT4zeygEAE9qHeg4oEolIkgoKCiRJBw4cUDweV3V19fA28+fP1/Tp07Vnz57zfo/+/n5Fo9ERFwDA5Oe7gBKJhO666y6tWLFCCxYskCS1tLQoPT1d+fn5I7YtKSlRS0vLeb9PbW2twuHw8KWiwt/LJQEAE4vvAqqpqdHhw4f1+OOPf6gFbNmyRZFIZPjS1NT0ob4fAGBi8PVG1E2bNum5557T7t27NW3atOHPl5aWamBgQJ2dnSMeBbW2tqq0tPS83ysUCikUCvlZBgBgAjM9AvI8T5s2bdJTTz2lF198UZWVlSO+vmTJEqWlpWnHjh3Dn6urq1NjY6OWL18+OisGAEwKpkdANTU1evTRR/XMM88oNzd3+HmdcDiszMxMhcNhffGLX9TmzZtVUFCgvLw8felLX9Ly5ct5BRwAYARTAT300EOSpFWrVo34/COPPKJbbrlFkvR3f/d3CgaD2rBhg/r7+7V27Vr9/d///agsFgAweQQ8z/M30W6MRKNRhcNhrXr2TqVmJ//cUP2r0z54o3cJdfh7DUZOo/2Q9ZTZh0/2TU2YM4GyPnMmMWRfmyTddcWL5sy/npltzoTT7NepID32wRudR13UPrzzdF+WOZPlY5Dr8VMF5kz/qUxzRpKCffbbRtjH0NgzJ+3DPqdW2Pdzqr7QnJGkjHb7cQgM2ffTO99+jmfm9Nt3JKn3ZI45s2jRcdP28diAtq//34pEIsrLu/DQZ2bBAQCcoIAAAE5QQAAAJyggAIATFBAAwAkKCADgBAUEAHCCAgIAOEEBAQCcoIAAAE5QQAAAJyggAIATFBAAwAlffxH1Yqg/XqpgZkbS28+8vNm8j/Ydl5gzkhSt8jc92ip7VsScycmwT8jNSB00ZyTpyRNXmDN/PuO35sw/ti02ZzYU7jdnJGl93iFz5h9aPmXOdPRlmzM3zztgzrRX5pozkvTrF+w/2+5Y8rfXD+NMxH7s8t/wd5uNXtNjzgx1ppszae/Y/yp0bzjNnJGk1UsPmzPNvReeaH0+g15yj214BAQAcIICAgA4QQEBAJyggAAATlBAAAAnKCAAgBMUEADACQoIAOAEBQQAcIICAgA4QQEBAJyggAAATozbYaQaCpy9JKnpYLl5F+EVp8wZSUr/VaE5k3HGM2e6FtkHKJ5stK8tv6TLnJGkivxOc2Zb4yfNmW9U/ZM58w/N9gGhknRPxXPmzH8osg8wTVHCnPnr19ebMzOmnDFnJCm1237ueW/Yh4SmDpkj6p1uvy11Xmo/3pKU0phpzuSfsB+7nlL7dfJS7BlJ+u1z9uG+aUts59FQT3JDkXkEBABwggICADhBAQEAnKCAAABOUEAAACcoIACAExQQAMAJCggA4AQFBABwggICADhBAQEAnKCAAABOjNthpCm5AwpmJd+PXrZ9AODpd/LNGUkKp9n31XaVj2GDb+eZM4Wv2dcWjE8xZyTp1U/mmjPhV9PMmf8+9b+aMxUrTpgzknT9P37ZnAkUDPjal1XoiH0w5vHBfH87S/eRsZ96mnpw0JxpzrLfbeUdM0ckSfE8+5VKj9pv67GrY+bM1Nxec0aS2mW/vQeGbI9VhpLcnkdAAAAnKCAAgBMUEADACQoIAOAEBQQAcIICAgA4QQEBAJyggAAATlBAAAAnKCAAgBMUEADACQoIAODEuB1G6rVnyMvISHr7tEvsw/xCuX3mjCRFEvYhnFlN9kM9mGMfahiZbY5o9mMRe0hSrCzfnMlrsg+fLHx9yJxJeTbfnJGk4nn24ZOpffbJnTlv95gzwZ4Oc8avrrn55kykKsWc6c+3/w6cSLffLrLbEuaMJMVkv05t19jP8ZSmbHOm+Kp2c0aSTncXmjOzLrWde/HYgI4msR2PgAAATlBAAAAnTAVUW1urq666Srm5uSouLtb111+vurq6EdusWrVKgUBgxOWOO+4Y1UUDACY+UwHt2rVLNTU12rt3r55//nnF43GtWbNGsdjI519uu+02NTc3D1/uv//+UV00AGDiMz0zvn379hEfb9u2TcXFxTpw4IBWrlw5/PmsrCyVlpaOzgoBAJPSh3oOKBI5++qpgoKCEZ//+c9/rqKiIi1YsEBbtmxRT8+FX/HT39+vaDQ64gIAmPx8vww7kUjorrvu0ooVK7RgwYLhz3/+85/XjBkzVF5erkOHDunrX/+66urq9Mtf/vK836e2tlb33Xef32UAACYo3wVUU1Ojw4cP66WXXhrx+dtvv3343wsXLlRZWZlWr16tY8eOadasWe/5Plu2bNHmzZuHP45Go6qoqPC7LADABOGrgDZt2qTnnntOu3fv1rRp095322XLlkmS6uvrz1tAoVBIoVDIzzIAABOYqYA8z9OXvvQlPfXUU9q5c6cqKys/MHPw4EFJUllZma8FAgAmJ1MB1dTU6NFHH9Uzzzyj3NxctbS0SJLC4bAyMzN17NgxPfroo/rMZz6jwsJCHTp0SHfffbdWrlypRYsWjckVAABMTKYCeuihhySdfbPpH3vkkUd0yy23KD09XS+88IIefPBBxWIxVVRUaMOGDfrmN785agsGAEwO5v+Cez8VFRXatWvXh1oQAOCjYdxOw/7YFceUlp38lOGXG+2vnPvUzHpzRpKOZNrfZNtWlGPODESSnwZ+TqjZ/iN985Y8c0aS8pIZd/suvQX26cLti+3XKXTG3wtbEmn2TNw+yFhZ79jfgtd1Wb45E5llP96SlP2OfeJ0Wrc942cadjBu30/7In93dWn2oeX+BO3X6UiTvzf7J7Ltk8GPvGPbV6Inub80wDBSAIATFBAAwAkKCADgBAUEAHCCAgIAOEEBAQCcoIAAAE5QQAAAJyggAIATFBAAwAkKCADgBAUEAHBi3A4j/cO/zVIwI/lhnJ6PmYu/ji60hyQVXNJpzhSHu82ZgZxec6ZoTsycee0Nf38CPTLPPkDRC9kHIQazB82ZAfvSJEkpTT4GwHYGzJnmq+0TTOP2ebYK2g+dJKltxZA5k/OW/e6ke+6AOZNXZD/HYzH7z1WS8qZ0mTPxHvu+AgX2E7an3ccUXEmBhP18vaSo07T9YKxfx5PYjkdAAAAnKCAAgBMUEADACQoIAOAEBQQAcIICAgA4QQEBAJyggAAATlBAAAAnKCAAgBMUEADAiXE3C87zzs5ESvT12XI+ZsElZJ9LJklDPf3mzGA8bt/PkP1KxT37bK1Er+1YnxMYsM+U8hI+jnnAx0Azn7PgAj4OxVC//Tj4Wd+Qj1urZx/pJklK9NqP+VC/fYGJXvv56uf2l+gxR87uK92+ryH7CEcFfJxCiV4fd3qSAkP2nQ3GbMdhsOfsz/Xc/fkF1+J90BYX2YkTJ1RR4W84JgBg/GhqatK0adMu+PVxV0CJREInT55Ubm6uAu/6tSAajaqiokJNTU3Ky8tztEL3OA5ncRzO4jicxXE4azwcB8/z1NXVpfLycgWDF36mZ9z9F1wwGHzfxpSkvLy8j/QJdg7H4SyOw1kch7M4Dme5Pg7hcPgDt+FFCAAAJyggAIATE6qAQqGQ7r33XoVCIddLcYrjcBbH4SyOw1kch7Mm0nEYdy9CAAB8NEyoR0AAgMmDAgIAOEEBAQCcoIAAAE5MmALaunWrZs6cqYyMDC1btky///3vXS/povv2t7+tQCAw4jJ//nzXyxpzu3fv1rXXXqvy8nIFAgE9/fTTI77ueZ7uuecelZWVKTMzU9XV1Tp69KibxY6hDzoOt9xyy3vOj3Xr1rlZ7Bipra3VVVddpdzcXBUXF+v6669XXV3diG36+vpUU1OjwsJC5eTkaMOGDWptbXW04rGRzHFYtWrVe86HO+64w9GKz29CFNAvfvELbd68Wffee69efvllLV68WGvXrlVbW5vrpV10l19+uZqbm4cvL730kusljblYLKbFixdr69at5/36/fffrx/+8Id6+OGHtW/fPmVnZ2vt2rXqMw60He8+6DhI0rp160acH4899thFXOHY27Vrl2pqarR37149//zzisfjWrNmjWKx2PA2d999t5599lk9+eST2rVrl06ePKkbbrjB4apHXzLHQZJuu+22EefD/fff72jFF+BNAEuXLvVqamqGPx4aGvLKy8u92tpah6u6+O69915v8eLFrpfhlCTvqaeeGv44kUh4paWl3g9+8IPhz3V2dnqhUMh77LHHHKzw4nj3cfA8z9u4caN33XXXOVmPK21tbZ4kb9euXZ7nnf3Zp6WleU8++eTwNkeOHPEkeXv27HG1zDH37uPgeZ73qU99yvvyl7/sblFJGPePgAYGBnTgwAFVV1cPfy4YDKq6ulp79uxxuDI3jh49qvLyclVVVekLX/iCGhsbXS/JqYaGBrW0tIw4P8LhsJYtW/aRPD927typ4uJizZs3T3feeac6OjpcL2lMRSIRSVJBQYEk6cCBA4rH4yPOh/nz52v69OmT+nx493E45+c//7mKioq0YMECbdmyRT09Pv8uxRgZd8NI3+3UqVMaGhpSSUnJiM+XlJTojTfecLQqN5YtW6Zt27Zp3rx5am5u1n333adrrrlGhw8fVm5uruvlOdHS0iJJ5z0/zn3to2LdunW64YYbVFlZqWPHjukb3/iG1q9frz179iglxd/fjhnPEomE7rrrLq1YsUILFiyQdPZ8SE9PV35+/ohtJ/P5cL7jIEmf//znNWPGDJWXl+vQoUP6+te/rrq6Ov3yl790uNqRxn0B4d+tX79++N+LFi3SsmXLNGPGDD3xxBP64he/6HBlGA9uuumm4X8vXLhQixYt0qxZs7Rz506tXr3a4crGRk1NjQ4fPvyReB70/VzoONx+++3D/164cKHKysq0evVqHTt2TLNmzbrYyzyvcf9fcEVFRUpJSXnPq1haW1tVWlrqaFXjQ35+vubOnav6+nrXS3Hm3DnA+fFeVVVVKioqmpTnx6ZNm/Tcc8/pN7/5zYg/31JaWqqBgQF1dnaO2H6yng8XOg7ns2zZMkkaV+fDuC+g9PR0LVmyRDt27Bj+XCKR0I4dO7R8+XKHK3Ovu7tbx44dU1lZmeulOFNZWanS0tIR50c0GtW+ffs+8ufHiRMn1NHRManOD8/ztGnTJj311FN68cUXVVlZOeLrS5YsUVpa2ojzoa6uTo2NjZPqfPig43A+Bw8elKTxdT64fhVEMh5//HEvFAp527Zt815//XXv9ttv9/Lz872WlhbXS7uo/vIv/9LbuXOn19DQ4P32t7/1qqurvaKiIq+trc310sZUV1eX98orr3ivvPKKJ8l74IEHvFdeecV7++23Pc/zvO9973tefn6+98wzz3iHDh3yrrvuOq+ystLr7e11vPLR9X7Hoaury/vKV77i7dmzx2toaPBeeOEF74orrvDmzJnj9fX1uV76qLnzzju9cDjs7dy502tubh6+9PT0DG9zxx13eNOnT/defPFFb//+/d7y5cu95cuXO1z16Pug41BfX+995zvf8fbv3+81NDR4zzzzjFdVVeWtXLnS8cpHmhAF5Hme96Mf/cibPn26l56e7i1dutTbu3ev6yVddDfeeKNXVlbmpaene5dccol34403evX19a6XNeZ+85vfeJLec9m4caPneWdfiv2tb33LKykp8UKhkLd69Wqvrq7O7aLHwPsdh56eHm/NmjXe1KlTvbS0NG/GjBnebbfdNul+STvf9ZfkPfLII8Pb9Pb2en/xF3/hTZkyxcvKyvI++9nPes3Nze4WPQY+6Dg0NjZ6K1eu9AoKCrxQKOTNnj3b++pXv+pFIhG3C38X/hwDAMCJcf8cEABgcqKAAABOUEAAACcoIACAExQQAMAJCggA4AQFBABwggICADhBAQEAnKCAAABOUEAAACcoIACAE/8f+dWcQaz8QTkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "DIGIT = 0  # Change me to be an integer from 0 to 9.\n",
    "LAYER = 1  # Layer 0 flattens image, so no weights\n",
    "WEIGHT_TYPE = 0  # 0 for variable weights, 1 for biases\n",
    "\n",
    "dense_layer_weights = model.layers[LAYER].get_weights()\n",
    "digit_weights = dense_layer_weights[WEIGHT_TYPE][:, DIGIT]\n",
    "plt.imshow(digit_weights.reshape((HEIGHT, WIDTH)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Did you recognize the digit the computer was trying to learn? Pretty trippy, isn't it! Even with a simple \"brain\", the computer can form an idea of what a digit should be. The human brain, however, uses [layers and layers of calculations for image recognition](https://www.salk.edu/news-release/brain-recognizes-eye-sees/). Ready for the next challenge? <a href=\"https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/images/mnist_linear.ipynb\">Click here</a> to super charge our models with human-like vision."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bonus Exercise\n",
    "\n",
    "Want to push your understanding further? Instead of using Keras' built in layers, try repeating the above exercise with your own [custom layers](https://www.tensorflow.org/tutorials/customization/custom_layers)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright 2022 Google Inc.\n",
    "Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at\n",
    "http://www.apache.org/licenses/LICENSE-2.0\n",
    "Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
